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Abstract Body 



Background / Context: 

A central focus within the Institute of Education Sciences (lES) is on the development 
and validation of instructional approaches and interventions that improve student achievement. 
Despite increased interest by lES, there is compelling evidence from the What Works 
Clearinghouse (WWC) that suggests just a handful of curricular programs are evaluated under 
rigorous experimental conditions. For example, of the 72 early mathematics programs reviewed 
by the WWC, seven met the clearinghouse’s evidence standards. Of the seven programs, just one 
demonstrated potentially positive effects for improving math outcomes. 

For establishing evidence-based mathematics programs, researchers have considered 
curriculum development as a design science (Brown 1992; Chard et ah, 2008; Clements, 2007). 
That is, curriculum developers engineer complex instructional interventions through an iterative 
process that incorporates a sequence of design-analysis-redesign cycles or phases (Shavelson, 
Phillips, Towne, & Feuer, 2003). Clements’ (2007) proposed “curriculum research framework’’ 
(CRF) highlights the importance of this science and explains the impact of anchoring math 
programs to the converging knowledge base of effective math instruction. Clements’ framework 
hypothesizes that curricula developed through a scientific process can produce significant effects 
under authentic educational conditions. 

The CRF (2007) includes 10 research-based phases of the curriculum development 
process. It calls for developers to subject programs to each phase of the framework, ensuring that 
research is present across all 10 phases. In Phase 8, for example, researchers use multiple data 
sources to address research questions such as: To what degree is the prescribed schedule of use 
adhered to and positively viewed by the teacher and students? 

Given the vital importance of using a scientific approach for curriculum development, we 
employed a design experiment methodology (Brown, 1992; Shavelson et ah, 2003) to develop 
and evaluate, FUSION, a first grade mathematics intervention intended for students with or at- 
risk for mathematics disabilities. We also expanded the development of FUSION to include 
elements from Clements’ (2007) framework. FUSION, funded through lES (Baker, Clarke, & 
Fien, 2008), targets students’ understanding of whole number concepts and skills and is being 
designed as a Tier 2 intervention for schools that use a multi-tiered service delivery model, such 
as Response to Intervention (Rtl). In developing this intervention, we have drawn extensively 
from the converging knowledge base of effective math instruction (Gersten et ah, 2009; National 
Math Advisory Panel, [NMAP] 2008) and the critical content areas of first grade mathematics 
recognized by national bodies (Common Core State Standards for Mathematics, [CCSS-M] 
2010 ). 

Purpose / Objective / Research Question / Focus of Study: 

Guiding the FUSION project are three primary objectives: (1) develop a 60-lesson 
intervention program that fosters students’ procedural fluency and conceptual understanding of 
whole number concepts, (2) assess the feasibility of the FUSION intervention and (3) assess the 
potential efficacy of the intervention in a subsequent randomized efficacy trial. 

This presentation will focus on FUSION’S initial feasibility study. We examined whether 
FUSION lessons were feasible for the teachers and students for which they are designed. We 
conceptualized feasibility as the ability for teachers to (a) implement the program as intended, 

(b) adhere to specific time parameters (i.e., 30 minute lessons), (c) use small-group 
implementation guidelines, (d) manage a variety of math models during instructional activities, 



SREE Fall 2011 Conference Abstract Template 



A-1 




and (e) deliver effeetive math instruetion. The study also targeted growth in student math 
outeomes to demonstrate the potential promise of the intervention. Under Clements’ (2007) CRT 
framework, our present researeh would map onto Phase 8: Formative Research - Multiple 
Classrooms. We will present student performance data, implementation fidelity data, and results 
from classroom observations that measured the quality of math instruction. In addition, we will 
describe FUSION’S early phases of development and evaluation (i.e.. Phases 1-7 of the CRF). 
Implications for future curriculum development and research will be discussed. 

Setting: 

The feasibility study took place in seven schools in two suburban school districts located 
in the northwest. Several schools within both districts receive Title- 1 funding. One district 
enrolls approximately 10,850 students: 17.4% receive special education services, 5.9% are 
English learners, 59.8% are eligible for free/reduced lunch, and 25.5% are minorities. The other 
district enrolls approximately 5,800 students: 19.6% receive special education services, 3.1% are 
English learners, 57% are eligible for free/reduced lunch, and 24.1% are minorities. 

Population / Participants / Subjects: 

Eight teachers (1 male) with varying experience levels participated in the feasibility study. 
They represented a range of teaching roles: 1 general education teacher, 5 special education 
teachers, and 2 educational assistants. Participants in this study included 39 first grade students, 
of which 23 were females. Student participation was based on: (a) teacher recommendations of 
students’ overall math proficiency and (b) students’ scores on two curriculum-based measures 
(CBM) of early mathematics. 

Intervention / Program / Practice: 

FUSION curriculum. The FUSION program is a Grade 1 (Tier 2) mathematics 
intervention that focuses specifically on building students’ early knowledge of whole number 
concepts. Four math strands comprise the program: (a) base- 10/place value, (b) basic number 
combinations, (c) multi-digit addition and subtraction without renaming, and (d) word problems. 
Each strand reflects the critical content of first grade mathematics (CCSS-M, 2010; NCTM, 
2006) and aligns with the recommendations of the NMAP (2008) and other experts in the field 
(Kilpatrick, Swafford, & Findell 2001; Wu, 2009). FUSION’S 60 scripted lessons utilize an 
explicit instructional format. Lessons contain teacher modeling, scaffolded instructional 
examples, and opportunities for academic feedback. Lessons incorporate a variety of math 
models and offer frequent opportunities for student practice and judicious review. A sample 
lesson is presented in Appendix B (Figure 1). 

Teachers were encouraged to complete one lesson per day, three times per week. Lessons 
lasted approximately 30 minutes and were delivered in small-group instructional formats, with 
approximately 4-5 students per group. FUSION instruction occurred outside of students’ core 
math and reading time. 

Professional development. Prior to the study, participating teachers received four hours 
of professional development in early mathematics instruction. This session focused on three key 
elements: (a) the research-based principles of math instruction, (b) the instructional design and 
delivery features of the FUSION curriculum, and (c) an overview of lessons 1-30. In the session, 
participating teachers were provided opportunities to deliver sample lessons and receive 
feedback on their teaching from the project staff. Teachers also learned how to administer all 
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student assessments. Midway through the study, all teachers participated in a four-hour follow- 
up workshop. A central focus of this session was previewing lessons 31-60 of the curriculum. 
During the study, all teachers received on-going professional development in the classroom (i.e., 
expert coaching). 

Research Design: 

In this development project, we use a design experiment methodology (Brown, 1992; 
Shavelson et ah, 2003) to develop a complex, feasible math intervention that is positioned for a 
Goal 3 efficacy trial in the lES goal structure. Design experiments offer a methodological 
structure for refining and developing instructional interventions through iterative cycles of 
development, observations, analysis, and refinement. Development of FUSION also included 
elements from Clements’ (2007) framework. Formative evaluation of FUSION is taking place 
across three implementation studies: (a) Brief Feaming Trials Study, (b) Feasibility Study, and 
(c) Pilot Study. The present research is the Feasibility Study of Project FUSION. The Pilot Study 
will be conducted during the 201 1-2012 school year. 

Data Collection and Analysis: 

We provide information on the data collected to date and the analyses planned for the 
presentation. 

Project staff collected observation data within each small group. Direct observations of 
teaching were conducted to provide preliminary evidence that teachers can feasibly implement 
FUSION in authentic educational settings. Instruction was also observed and evaluated through 
teacher-recorded videos. Data obtained from the observations helped us examine elements of 
feasibility (e.g., adherence to specific time parameters) and usability (e.g., amount of teacher 
scripting). Observations were coded using the FUSION observation instrument (see Figure 2 in 
Appendix B). The instrument contains two sections. The first section measures implementation 
fidelity of the FUSION program. After each activity (range 4-5 per lesson), observers rated the 
fidelity of teacher’s implementation. Observers rated implementation fidelity for each activity 
using a 0-2 scale (0 = not taught, 1 = partial implementation, and 2 =full implementation). An 
overall fidelity score for each observation was calculated by averaging ratings across the 
activities. 

The second section of the observation instrument measures the quality of classroom 
instruction (11 items) across three domains: learning environment, classroom management, and 
the delivery of instruction. We are targeting teachers’ delivery of effective math instruction to 
demonstrate the potential promise of the intervention. At the conclusion of each observation, 
observers recorded their overall impressions of 1 1 features of instructional quality using a 4- 
point holistic rating scale (1 = not present, 2 = somewhat present, 3 = present, and 4 = highly 
present) Previous work documented the instrument’s capacity to predict student math outcomes 
(Doabler et ah, in preparation). Teachers also completed instructional logs following each lesson, 
providing key information on lesson usability. Teachers’ professional feedback was utilized to 
revise critical features of the FUSION program (e.g., determining the number of instructional 
examples in activities). At the end of the study, teachers will complete demographic, perception 
and feasibility surveys. 

We are targeting student learning gains to demonstrate the potential promise of Fusion. 
Student performance was assessed using a set of measures that are considered proximal to the 
FUSION intervention: ProFusion and two CBMs of early mathematics. All measures were group 
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administered. ProFusion, a researcher-developed assessment, was used to measure students’ 
knowledge of whole number concepts and skills. The measure is comprised of 40 items that 
assess students’ knowledge of place value, simple addition and subtraction word problems, 
multi-digit addition and subtraction, and basic number combinations. Modified versions of the 
Quantity Discrimination (QD) and Missing Number (MN) measures (Clarke & Shirm, 2004) 
were also used to assess the potential impact of the program. The QD measure requires 
participants to circle the larger of two numbers. For the MN measure, participants write in the 
missing number from a string of numbers between 0 and 10 (e.g., 3 _ 5). FUSION teachers 
administered all measures at pretest (fall) and posttest (spring). Project staff scored all student 
assessments. 

Findings / Results: 

Preliminary analyses indicate that teachers are meeting acceptable levels of implementation 
fidelity. The average fidelity score from the first set of observations was 1.44 (SD = .22). Scores 
for the quality of instruction were: learning environment (M = 3.29, SD = .08) classroom 
management (M = 3.06, SD = .27), and delivery of instruction (M = 2.95, SD = .10). While 
preliminary, these findings indicate that teachers can feasibly implement FUSION as intended. 
Inter-rater reliability was (72%) for the implementation fidelity and (82%) for the quality of 
instruction. 

Overall, teachers have expressed encouraging views of FUSION. Anecdotal records from 
classroom visits and professional development sessions indicate that teachers are satisfied with 
fusion’s overall structure and comfortable with lesson implementation. Teachers report that 
students are benefiting from the program and in particular building conceptual knowledge and 
procedural fluency in whole numbers. 

For student performance data, we plan to investigate student gains across the academic 
year. We expect to find strong relationships between FUSION and changes in CBM and 
ProFusion scores. These preliminary findings would serve as promise of FUSION for positively 
influencing student math outcomes. Analyses are currently underway. 

Conclusions: 

Persistent problems in student mathematics achievement have become a national concern. 
This attention is most visible among the math performances of children from low-income 
backgrounds, minorities, and students with or at risk for math difficulties. Leading 
mathematicians, educators, and expert panels call for the development of curricula that are 
coherent, rigorous, and reflect the converging knowledge base of math intervention research. 
According to Clements’ (2007) framework, researchers should use 10 research-based phases to 
develop and evaluate programs. FUSION was developed through sequence of design-analysis- 
redesign phases. Though preliminary, our findings indicate that teachers can feasibly implement 
FUSION in small-group settings. 
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Appendix B. Tables and Figures 

Figure 1 

Sample lesson from FUSION program 



Activity 


Strand 


Objective 


Warm Up 


NC 


Flashcard game (subtraction = 0] 


1 


PV 


Count by ones to 100 


2 


PV 


Identify and sequence numbers 91-100; Find missing number 91-100 


3 


PV 


Decompose and write numbers 14-95 from expanded notation 


4 


PV 


Greater than / less than 


Wrap Up 


NC 


Math facts minus 1 timed practice 



Materials: Flashcards (subtraction facts = 0), hundreds chart, numeral cards 91-100, teacher 
whiteboard, place value mat and 10 ten sticks and 9 cubes per student, flashcards 1-1 
to 10-1, minus 1 subtraction facts worksheet and cumulative worksheet, pencils 

Vocabulary: Tens, ones, greater than, less than, more, less, subtraction, minus 





Group Responses 




Math Talk 


Possible Error 

A 


Monitor for 
Understanding 

✓ 




Warm Up: Flashcard game 


5 minutes 



Materials: Flashcards (subtraction = 0) 







o We’re going to do problems that subtract the same number. Tell your 
partner how to figure out the answer to a problem like 4 - 4. 

Monitor math talk. Call on a student to offer the idea that the answer is always 0. 



o When we subtract a number from itself, the answer is always 0. What 
does 4-4 equal? (0] 



Play the flashcard game: 

o Allow each student 3 seconds per fact, 
o If correct, give card to the student. 

o If incorrect call on another student to provide the correct 
answer, then review the strategy (see below], and return the 
card to the pile for another turn. 

Strategy correction: 



4 
- 4 
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o 4-4 {when we subtract a number from itself, the answer is always O') 

• If time permits, have students count and report the number of facts correctly 
answered. 



1. Count by ones to 100 

Materials: Hundreds chart 

• Show the hundreds chart. 

o In our last lesson, you counted all the way to 100! You're going to count 
by ones to 100 starting with 91. 

o What number do we start with? (91] What number do we end with? 

( 100 ] 

o Count by ones from 91 to 100 as I touch the numbers on the hundreds 
chart. Ready, count. 

• After students have counted, turn the hundreds chart over so they can’t see it. 

o Now count from 91 to 100 without looking at the hundreds chart. 

• Repeat until firm. 

o Now count from 51 to 100 without looking at the hundreds chart. 



CORRECT RESPONSE 


^STUDENT ERRORS 


Wow! You counted all the way 
to 100! Tomorrow you’ll 
count to 100 beginning with 
1! 


If students make an error, stop and model the 
counting sequence beginning with a few numbers 
before the error through a few numbers after the 
error. Practice with the students until they can 
count this short sequence correctly by themselves. 
Go back to 51 and repeat to 100 until firm. 



2. Identify and sequence numbers 91-100; Find the missing number 91-100 



Materials: Number cards 91-100 

• Show number cards 91-100 in random order. Have group identify each number. 

o What number? 



CORRECT RESPONSE 


/j^TUDENT ERRORS 


Right, this number is [97]. Place the 
number card on the table. 


This number is [97]. What number? (97] 
Yes, 97. Keep number card for another turn. 
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• Distribute cards so each student has several numbers. 

o Now let’s put the numbers in order. What are we going to start with? [9 1] 

Have the student who has 91 place it on the table to the left of the group. 

o What number goes next? 

• Continue until all the cards have been placed in order. 

o Let’s see if all the numbers are in order. Let’s count them starting with 
91. 

• Touch each number as students identify 91-100. 

• Play the "Missing Number Game." 

o Have students close their eyes while you turn a card over. 

o Have students open their eyes and put their thumb up when they know the 
missing number. 

• Repeat with 3-4 more "missing numbers.” 

• Have the group count from 91-100 without looking at the numbers. 

3. Decompose and write numbers through 99 from expanded notation 

Materials: Teacher place value whiteboard; Place value mat, 10 ten sticks, 9 cubes per 
student 

Vocabulary: Tens, ones 

o You have been doing such a great job that we’re going to make models 
for lots of numbers that you know. 

• Write on the board: 90 

o Everyone, what number? [90] 
o Ninety has how many tens ? (9] How many ones ? [0] 
o Your turn. Use your ten sticks and cubes to show me 90. 

• Monitor as students place the models in the correct columns. 

o Everyone, count the ten sticks and cubes to make sure we have 90. 

• Add "+ 3” on the board: 90 + 3 

o Now want you to show me what 90 + 3 looks like, 
o How many tens ? [9] How many ones ? (3] 

o Your turn. Use your ten sticks and cubes to show me 90 plus 3. 

• Monitor as students place the models in the correct columns. 

o Does anyone know what number 90 + 3 equals? 

• Call on an individual student. Confirm that 90 + 3 is 93. 
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o 



Everyone, count the ten sticks and cubes to make sure we have 93. 



o Now write 93 on your board. Make sure to put the numbers in the 
correct columns. 

• Monitor that students write 93 in the correct place value columns. 

4. Greater than / less than 



Materials: Teacher white board 

Vocabulary: greater than, less than, more, less 

o Today we're going to learn how to find out if a number is greater than or 
less than another number. 

• Write on the board: > < 

o We use this symbol (write on the board: >] for greater than and this 
symbol (write on the board: <] for less than. 

• Write on the board: 7 4 

o You already know which of these numbers is more and which is less , 
o Which number is more? (7] Yes, 7 is more or greater than 4. 
o Which number is less? (4] Yes, 4 is less than 7. 

o Now 1 look at the first number, which is 7. Is 7 greater than or less than 

4? (Greater than] 

o Since it is greater than, 1 write this symbol for greater than (write > 
between the numbers]. This says 7 is greater than 4. 

o Say that with me. (7 is greater than 4] 

o You can see that the big side of the arrow faces the big number. That will 
help you remember which way the arrow should point. 

o Say this by yourself. (7 is greater than 4] 

o Let’s do another one. 

• Write on the board: 2 8 

o Which number is more? (8] Which number is less? (2] 

o Now 1 look at the first number, which is 2. Is 2 more than or less than 8? 

(Less than] 

o Since it is less than, 1 write the symbol for less than (write < between the 
numbers]. This says 2 is less than 8. 

o Say that with me. (2 is less than 8] 

o See how the small side of the arrow faces the small number? That will 
help you remember which way the arrow should point. 
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o Say this by yourself. [2 is less than 8] 



Wrap Up: Math facts minus 1 timed practice 



Materials: Minus 1 facts worksheet, and minus 1 eumulative worksheet, pencils 
Vocabulary: Subtraction, minus 

• Give worksheets with minus 1 fact problems if students did not meet criteria in the 
previous lesson, or minus 1 cumulative worksheet to students who did meet the 
criteria. 

• Pass out pencils and give the following directions. 

o Look at the problems on your worksheet. Remember to use your 
thinking strategies to solve each problem. 

• If time allows you may review the strategy for each type of problem and/or have 
students orally solve a few problems in the first row on their worksheet. 

o Today you’re going to write the answer to these problems. You will have 
one minute to answer as many problems as you can. Try your best. 

• Tell students to begin and set a timer for 1 minute. Monitor that students aren’t 
using their fingers. After 1 minute, stop the students and collect their papers. 

• Note: If students only complete 1 or 2 rows, you may mark the last problem 
completed and use the same worksheet for the next timing. 

• After the lesson, correct the fact timings. Record how many problems each student 
completed correctly in 1 minute on the facts data sheet. Note if students are making 
errors using one of the strategies and precorrect in future lessons as needed. 







Teacher Note: In the next lesson, all students will move on to a new fact timing. If 
students have still not met eriteria, try to provide extra practice on these faets 
within and outside the lesson. 
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Figue 2 



FUSION Observation Instrument 




General Observation Features 



Features 

1 . Community of positive learning 
(Rapport, Respect, Positive Attitude) 

2. Organization of instruetional materials and learning tasks 
(Preparation, teacher-initiated transitions, accessibility) 

3. Support of students’ emotional needs (Sensitivity & Respect) 

4. Effective and efficient classroom management (Clear 
expectations, minimizes non-instructional time, addresses appropriate beh) 

5. Productive disposition of mathematical learning 
(Positive outlook on math, views math as important, confidence) 

6. Clear and consistent delivery of instruction (Teacher 
demonstrations, pacing, consistent language, minimizes student confusion) 

7. Student participation and engagement 
(Active involvement, compliance, completion of work) 

8. Checks of student understanding (Timely checks, active 
monitoring and feedback, individual and group response opportunities) 

9. Use of instructional adjustments (Student response time, 
accommodates learning needs, allows independent learning) 

10. Accomplishment of instructional tasks and activities 
(Scheduling, completion of tasks, use of instructional time, student routines) 

1 1 . Teaching for mathematical proficiency 

(States purpose of lesson, addresses big ideas, effective teaching examples, 
anticipates student misconceptions, frequent instructional interactions) 
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Fidelity of Implementation 


Activity: Level of Implementation Feasibility 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 


Activity: 


Notes: 


o o o 

Not 

Full Partial Taught 


Materials 
Y N 


Instr. Ex. 
Y N 
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